Reinforcement learning based on local state feature learning and policy adjustment

The extension of reinforcement learning (RL) to large state space has inevitably encountered the problem of the curse of dimensionality. Improving the learning efficiency of the agent is much more important to the practical application of RL. Consider learning to optimally solve Markov decision problems in a particular domain, if the domain has particular characteristics that are attributable to each state, the agent might be able to take advantage of these features to direct the future learning. This paper firstly defines the local state feature, then a state feature function is used to generate the local state features of a state. Also a weight function is introduced to adjust current policy to the actions worth exploring. Based on the above, an improved SARSA algorithm, Feature-SARSA, is proposed. We validate our new algorithm by experiment on a complex domain, named Sokoban. The results show that the new algorithm has better performance. 2003 Elsevier Science Inc. All rights reserved.